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Abstract 


This  study  analyzed  aircraft  inspection  data  to  determine  if  quality  enhancements 
were  realized  after  an  Action  Workout  (AWO)  was  accomplished.  Pretest  and  post-test 
assessment  data  from  three  separate  units  were  analyzed  to  determine  whether  overall 
quality  improvements  were  made. 

This  study  operationally  defined  quality  in  terms  of  Quality  Verification 
Assessment  ratings  compiled  before  and  after  each  AWO  event.  Comparisons  were 
made  to  determine  if  overall  quality  improved,  declined,  or  remained  unchanged. 
Parametric  t-tests  and  nonparametric  chi-square  analyses  were  used  to  determine  the 
significance  of  any  differences  between  the  pretest  and  post-test  data  sets. 

The  results  provide  plausible  evidence  that  quality  enhancements  can  be  realized 
as  a  result  of  Action  Workouts.  Results  at  two  of  the  three  units  analyzed  indicate  that 
overall  quality  of  major  aircraft  inspection  processes  improved  considerably,  possibly  as 
a  result  of  the  Action  Workout  intervention.  Results  at  the  third  site,  however,  remained 
essentially  unchanged. 

The  evidence  also  suggests  that  many  of  the  changes  and  improvement  ideas 
implemented  during  the  AWO  are  seemingly  intact  and  being  utilized  by  respective 
maintenance  personnel.  This  may  also  indicate  process  owner  "buy-in"  and  acceptance 
of  change,  two  essential  principles  of  quality  improvement.  This  research  establishes  a 
firm  foundation  for  future  research  efforts. 
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MEASURING  THE  LONG-TERM  EFFECTS  OF  ACTION  WORKOUTS 


I.  Introduction 


General  Issue 

Throughout  the  past  decade,  reduced  defense  spending  has  led  to  major  force 
reductions  in  the  military.  While  the  emphasis  on  spending  continues  to  escalate,  so  too 
does  the  demand  for  military  service.  Force  drawdowns  and  continued  demand  for 
military  services  have  resulted  in  the  military  having  to  do  "more  with  less."  Operations 
tempo  has  not  diminished  with  the  force  reductions  and  base  closings.  Peacekeeping 
missions  throughout  the  world  (e.g.  Bosnia,  Iraq,  and  Somalia)  have  demanded,  and  will 
continue  to  demand,  military  presence  for  an  uncertain  number  of  years  to  come.  General 
Richard  E.  Hawley,  Commander,  Air  Combat  Command  (ACC),  summed  up  the  current 
operating  environment  of  one  of  the  leading  USAF  Commands  in  a  speech  delivered  at 
the  1996  quality  symposium: 

One  of  our  goals  in  ACC  is  to  cut  our  cost  of  operations  by  30  percent. .  .But  the 
only  way  we  can  achieve  a  30  percent  reduction  in  the  cost  of  operating  is  by 
following  a  clearly  defined,  easy-to-follow  road  map.  That's  what  our  quality 
tools  give  us.  We  are  making  action  workouts  and  six  sigma  landmarks  on  our 
journey  to  continuous  improvement  in  Air  Combat  Command.  These  tools  are 
bringing  quality  to  life  for  our  people.  That's  what  "quality  Air  Force"  means: 
bringing  quality  to  life  for  our  people.  They're  helping  us  work  smarter,  rather 
than  harder.  We're  getting  our  work  done  faster  and  safer  by  doing  things  the  right 
way  once,  instead  of  over  and  over  again.  A  lot  of  the  tough  issues  we  face  -  high 
operations  tempo,  and  shrinking  budgets  among  them  —  can  be  addressed  by  the 
tools  called  action  workout  and  six  sigma.  (Hawley,  1996) 


1 


In  order  to  meet  the  challenges  of  maintaining  readiness  and  sustaining  increased 
operations  tempo  with  fewer  resources,  military  commanders  are  forced  to  operate  more 
efficiently.  Quality  management  philosophies  were  adopted  to  assist  commanders  in 
meeting  these  challenges.  In  the  United  States  Air  Force,  an  aggressive  quality 
improvement  tool  known  as  an  Action  Workout  (AWO)  is  being  used  in  conjunction  with 
quality  management  concepts  to  cut  costs  and  improve  key  processes.  Hope  among  the 
Air  Force  leadership  is,  this  will  ultimately  allow  for  a  leaner,  more  efficient  Air  Force 
(Action,  1997).  Air  Combat  Command's  Action  Workout  Team  was  awarded  Vice 
President  Gore's  Hammer  Award  for  excellence  in  government  in  July  1996  (Action, 

1 997).  The  Hammer  Award  is  presented  to  federal  employee  teams  "who  have  made 
significant  contributions  in  support  of  reinventing  government  principles"  (Action,  1997). 
Dramatic  success  stories  abound  and  the  Action  Workout  craze  has  quickly  become  a 
staple  in  the  Air  Force's  quality  culture  (DoD,  1996a:  1-3).  The  Air  Force  believes  that, 
in  addition  to  increasing  productivity  and  reducing  process  cycle  times,  they  will  also 
achieve  lower  total  costs  and  higher  levels  of  quality.  Many  stories  report  the  success  of 
AWO’s  in  terms  of  process  savings,  but  are  these  savings  coming  at  the  expense  of 
overall  quality?  The  purpose  of  this  study  is  to  investigate  this  critical  question  and 
determine  whether  reduced  cycle  times  are  realized  at  the  expense  of  the  overall  quality 
of  the  process. 

Action  Workout  Defined 

According  to  The  Quality  Approach,  the  Air  Force's  handbook  on  quality,  an 
Action  Workout  is  defined  as  "a  rapid,  concentrated,  high-energy,  team  effort  to  make 
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dramatic  productivity  improvements  in  any  organization"  (DoD,  1996b:  41-45).  Action 
Workouts  focus  on  eliminating  non-value  added  work  and  are  designed  to  reduce  cost, 
work  hours,  and  cycle  times  of  key  processes.  The  seven  primary  sources  of  waste  and 
inefficiency  that  constitute  non-value  added  work  are  rework,  inventory,  waiting,  over 
production,  excess  motion,  redundant  processing,  and  transportation  and  conveyance 
(DoD,  1996b;  41-45).  Action  Workouts  consist  of  a  five-step  process:  (1)  Identify  A 
Candidate  Process,  (2)  Conduct  Site  Visit,  (3)  Unit  Preparation  and  Data  Review,  (4) 
Action  Workout  Event,  and  (5)  Follow-on  Actions  (DoA,  1996:  4). 

Step  1:  Identify  A  Candidate  Process.  A  candidate  process  for  an  A  WO  must 
provide  an  opportunity  to  radically  improve  cost,  defects,  waste,  or  overall  cycle  time. 
Opportunities  should  be  verified  based  on  a  review  of  pertinent  data  such  as  process 
performance  measures,  unit  strategic  plans,  and  operational  readiness  inspection  reports. 
"The  primary  consideration  should  be  high  potential  for  significant  gains  in  efficiency." 
After  a  process  is  identified,  the  unit  commander  formally  invites  the  headquarters  Action 
Workout  Team  to  begin  Step  2. 

Step  2:  Conduct  Site  Visit.  The  site  visit  allows  the  AWO  team  to  provide 
training  to  the  unit  on  AWO  concepts  and  gives  the  team  exposure  to  the  identified 
process.  Also,  the  critical  path,  or,  "long  pole  in  the  tent"  for  the  process  is  identified. 

The  critical  path  determines  what  key  subtasks  should  be  targeted  for  improvement. 

Step  3:  Unit  Preparation  and  Data  Review.  The  unit's  primary  objective  during 
this  step,  which  is  commonly  referred  to  as  the  pre-work,  is  to  collect  data  on  the  process 
in  its  current,  unaltered  state,  to  establish  a  baseline  on  which  all  improvements  will  be 
measured  during  the  AWO  event.  Baseline  data  consists  of,  but  is  not  limited  to,  cycle 
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times,  man-hours,  and  distance  traveled  for  each  subtask.  Units  use  videotapes  of  the 
process  and  key  subtasks  to  ease  the  collection  effort,  which  may  require  approximately 
two  to  three  weeks  to  accomplish. 

Step  4:  Action  Workout  Event.  The  actual  AWO  event  lasts  just  five  duty  days; 
no  more  and  no  less.  Process  owners  systematically  and  vehemently  attack  waste  by 
eliminating  non-value-added  steps  and  creating  innovative  changes.  A  unique  aspect  of 
the  event  is  "trystorming,"  which  means  physically  testing  every  improvement  idea  in  the 
process  and  documenting  the  results  of  the  improvements  as  the  event  goes  on. 
Trystorming,  as  opposed  to  brainstorming,  ensures  that  all  ideas  are  given  ample 
opportunity  for  incorporation  into  the  improvement  effort.  The  major  premise  is  to 
"accomplish,"  not  "discuss."  To  facilitate  radical  change,  the  AWO  team  brings 
headquarters  functional  policy  experts,  engineers,  and  technical  order  specialists  to  the 
event  to  clear  obstacles  and  make  on-the-spot  changes  to  policy  and  written  procedures. 
Activities  during  this  phase  embody  the  term  "Action  Workout." 

Step  5:  Follow-on  Actions.  The  process  improvement  follow-through  phase 
ensures  actions  that  could  not  be  completed  during  the  five-day  event  are  properly 
addressed.  A  list  of  actions  is  turned  over  to  the  unit  commander  for  his/her  action  over 
the  proceeding  thirty  days.  Progress  reports  and  results  are  forwarded  to  the  Commander, 
Air  Combat  Command,  for  final  review. 

The  Air  Combat  Command  Quality  and  Management  Innovation  Squadron 
describes  AWO  as  a  primary  tool  for  attacking  waste  in  critical  work  processes.  The 
AWO  philosophy  is  secured  in  the  belief  that  decreasing  process  cycle  time  can  attain 
significant  increases  in  productivity,  efficiency,  and  quality.  It  is  also  believed  that  as 
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process  cycle  times  are  reduced,  other  dimensions  of  work  processes,  such  as  cost  and 
quality,  are  forced  to  improve  as  well  (Action,  1997). 

AWO  Success  Stories 

In  1 997,  Air  Mobility  Command  reported  that  Action  Workouts  saved  $24  million 
(Hickey,  1997b).  An  AWO  conducted  at  Fairchild  AFB,  Washington,  for  example, 
produced  a  dramatic  50%  reduction  in  man-hours  required  to  process  reparable  aircraft 
parts  (Hickey,  1997a).  Mobility  processing  improvements  at  McGuire  AFB,  New  Jersey 
led  to  a  60%  reduction  in  unit  processing  times.  This  translated  into  a  man-hour  savings 
of  over  90%  (Hickey,  1997a).  The  AWO  conducted  on  the  B-l  Bomber  phase  inspection 
process  at  Dyess  AFB  resulted  in  the  elimination  of  approximately  100  man-hours, 

64,000  feet  of  travel  distance,  and  65  hours  of  cycle  time  (Hawley,  1996).  Other 
examples  include  cutting  the  F-15  phase  inspection  cycle  time  at  Mt  Home  AFB  from  7.1 
days  to  5.4  days,  a  24%  improvement  (Action,  1997). 

In  total,  Air  Mobility  Command  and  Air  Combat  Command  combined  to  perform 
more  than  40  Action  Workout  events  since  1996,  and  they  are  gearing  up  to  perform 
many  more  in  the  years  to  come  (Hickey,  1997a;  Action,  1997).  Proponents  of  AWO 
provide  compelling  evidence  for  tangible  process  improvements.  Less  compelling  is  the 
evidence  for  overall  improvements  in  quality. 

AWO's  in  Practice:  Phase  and  Isochronal  Aircraft  Inspections 

An  Action  Workout  can  be  implemented  in  virtually  any  process.  In  Air  Combat 
Command,  for  example,  a  process  is  a  candidate  for  an  AWO  if  a  need  exists  to  improve 
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cycle  time  (Action,  1997).  This  study  will  focus  on  Phase  and  Isochronal  (ISO)  aircraft 
inspection  processes.  These  major  scheduled  inspection  concepts  differ  only  in  terms  of 
scheduling  criteria  and  are  critical  to  ensuring  the  safety  and  airworthiness  of  major 
weapon  systems  in  the  Air  Force.  Phase  inspections  are  scheduled  on  the  basis  of  aircraft 
flying  time,  while  ISO  inspections  are  scheduled  according  to  average  daily  utilization 
rates  (DoD,  1 996a:  2-4).  The  scheduled  inspections  are  performed  in  accordance  with 
technical  order  requirements  and  consist  of  thorough  visual  and  non-destructive 
inspections  of  critical  areas,  systems,  and  structures.  Operational  checks,  time 
compliance  technical  orders,  and  other  special  inspections  are  also  performed.  Major 
inspection  processes  are  integral  to  preventive  maintenance  requirements  for  the  majority 
of  the  fleet. 

Inspection  process.  Maintenance  personnel  assigned  to  the  Inspection  Section  at 
each  of  the  identified  units  accomplish  major  (Phase/Isochronal)  aircraft  inspections.  The 
actual  composition  of  each  inspection  section  may  vary  according  to  unit  organizational 
structures,  but  typically  is  made  up  of  aircraft,  specialist,  and  weapons  systems 
maintenance  personnel.  Inspections  are  accomplished  in  four  phases:  (1)  Pre-inspection 
phase,  (2)  Look  Phase,  (3)  Fix  Phase,  and  (4)  Post-inspection  Phase. 

The  "pre-inspection  phase"  consists  of  pre-inspection  meetings,  aircraft 
preparation,  and  inspection  area  preparation.  Aircraft  washes  are  also  accomplished  prior 
to  the  actual  inspection,  which  is  performed  during  the  "look  phase"  of  the  process.  The 
inspection  is  accomplished  according  to  applicable  technical  order  procedures  and  all 
discrepancies  noted  during  the  look  phase  are  documented.  The  next  phase  is  referred  to 
as  the  "fix  phase."  The  fix  phase  consists  of  correcting  discrepancies  found  during  the 
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look  phase  and  accomplishing  preplanned  or  other  scheduled  maintenance.  The  "post¬ 
inspection"  phase  consists  of  performing  operational  checks,  and  if  required,  preparing 
the  aircraft  for  flight  (DoA,  1997:  120-122).  Depending  on  the  weapon  system,  major 
inspections  require  approximately  five  to  ten  duty  days  to  accomplish.  The  inspection 
section  dock  chief  must  notify  quality  assurance  upon  completion  of  the  inspection,  so 
that  an  assessment  of  the  inspection  can  be  made. 

Quality  Verification  Assessments.  According  to  ACCI  21-101,  Quality 
Assurance  is  required  to  perform  a  Quality  Verification  Assessment  (follow-up)  of  every 
major  aircraft  inspection  to  verify  that  the  inspection  was  properly  accomplished  (DoA, 
1997:  128).  All  discrepancies  noted  during  the  Quality  Verification  Assessment  (not  the 
actual  inspection)  are  documented  and  entered  into  the  Quality  Assurance  database. 
Discrepancies  are  classified  (by  Quality  Assurance)  as  either  major  or  minor  defects. 
Defects  that  may  lead  to  severe  damage  to  equipment  or  injury  to  personnel  are  classified 
as  major  defects.  An  example  of  a  major  defect  is  a  foreign  object,  such  as  a  screw,  bolt, 
or  a  piece  of  safety-wire,  found  in  an  engine  inlet.  A  minor  defect  is  a  missed  or 
incomplete  inspection  item  that  will  not  result  in  excessive  damage  to  equipment  or 
safety  of  flight.  Some  examples  of  minor  defects  are  tom  seat  cushions,  dirty  windows, 
or  loose  fasteners. 

Problem  Statement 

The  Action  Workout  motto,  "Radical  Improvement...Right  Now"  accurately 
describes  the  potential  effectiveness  of  this  quality  improvement  tool.  Action  Workout's 
have  been  proven  successful  in  dramatically  reducing  waste  and  cycle  times  in  many 
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critical  work  processes.  These  reductions  are  then  reflected  in  terms  of  man-hour  and 
cost  savings.  With  the  emphasis  being  placed  on  results,  this  "radical  improvement"  in 
reduced  cycle  time  may  only  be  realized  at  the  expense  of  secondary  processes  or 
measures,  possibly  degrading  the  overall  quality  of  the  process. 

Question 

The  research  question  in  this  study  is:  Does  an  Action  Workout  improve  the 
overall  quality  of  aircraft  major  inspection  processes? 

Objective 

The  objective  of  this  study  is  to  gather  and  analyze  aircraft  inspection  data  to 
determine  if  quality  enhancements  are  realized  after  an  action  workout  is  accomplished. 

To  obtain  a  representative  sample  of  inspection  results,  Quality  Verification 
Assessment  data  were  collected  from  three  separate  Quality  Assurance  inspection  teams 
who  conduct  maintenance  assessments/inspections  on  the  A-10,  EC/RC-135,  and  E-3 
weapon  systems  at  the  355  WG,  Davis  Monthan  AFB,  A Z,  the  55  WG,  Offutt  AFB,  NE, 
and  the  552  ACW,  Tinker  AFB,  OK,  respectively.  The  three  teams  were  chosen 
primarily  because  of  the  availability  of  data  and  represent  a  wide  array  of  inspection 
requirements  and  improvement  opportunities. 

This  research  will  operationally  define  quality  in  terms  of  Quality  Verification 
Assessment  ratings  gathered  before  and  after  each  Action  Workout  event.  Three 
potential  outcomes  are  possible  from  a  comparison  of  before  and  after  data:  (1)  Quality 
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Verification  Assessment  ratings  improve  after  the  intervention,  (2)  Quality  Verification 
Assessment  ratings  stay  the  same,  or  (3)  Quality  Verification  Assessment  ratings  decline. 
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II.  Literature  Review 


Introduction 

Action  Workouts  are  one  of  several  process  improvement  tools  the  Air  Force  has 
adopted  to  implement  "Quality  Air  Force,"  the  Air  Force's  approach  to  Total  Quality 
Management  (TQM).  Quality  Air  Force  is  defined  in  The  Quality  Approach  as  "a 
leadership  commitment  and  operating  style  that  inspires  trust,  teamwork,  and  continuous 
improvement  everywhere  in  the  Air  Force"  (DoD,  1996b:  1).  Other  process 
improvement  tools  include,  but  are  not  limited  to,  benchmarking,  process  reengineering, 
and  the  continuous  improvement  process  (DoD,  1996b:  31-54).  Action  Workouts  exhibit 
characteristics  of  Scientific  Management,  Total  Quality  Management,  and  Continuous 
Improvement.  The  literature  surrounding  these  concepts  is  explored. 

Scientific  Management 

In  the  early  1 900’s,  the  beginning  of  the  industrial  revolution,  continuous 
improvement  efforts  began  to  take  shape  under  the  umbrella  of  Scientific  Management. 
Fredrick  Taylor,  otherwise  known  as  the  Father  of  Scientific  Management,  initiated  the 
efforts  that  would  revolutionize  American  manufacturing.  Taylor  believed  in  using 
scientific  methods  to  find  the  "one  best  way"  to  produce.  He  also  believed  that  managers 
had  a  responsibility  to  provide  workers  with  necessary  resources  and  training  to  do  their 
job  (Evans  and  Lindsay,  1996:108-109).  Workers  were  told  what  to  do  and  how  to  do  it 
by  top  management,  who  held  all  decision-making  authority.  With  a  focus  on  efficiency 
and  productivity,  Scientific  Management  principles  made  the  United  States  the  world's 
leading  industrial  power  (Evans  and  Lindsay,  1996:  109).  According  to  Taylor, 
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Scientific  Management  is  based  on  four  principles:  (1)  Scientific  analysis  and 
documentation  of  work  methods,  (2)  Scientific  selection  of  workmen,  (3)  Bringing 
together  the  science  and  the  man,  and  (4)  Division  of  work  (Taylor,  1987:  72-75; 
Thurston,  1998:  3). 

In  the  first  principle,  "Scientific  analysis  and  documentation  of  work  methods," 
Taylor  advocated  using  scientific  methods  such  as  time  and  motion  studies,  to 
systematically  collect  the  "great  mass  of  traditional  knowledge"  from  workers,  and 
formulate  specialized,  standardized,  and  simplified  ways  to  perform  elements  of  work 
(Taylor,  1987:  72;  Thurston,  1998:  3). 

Action  Workouts  exemplify  this  principle  in  almost  every  regard.  Examples  exist 
in  the  first  four  steps  of  the  AWO  process.  The  very  first  step  of  the  AWO  process 
requires  process  owners  to  systematically  analyze  data  to  determine  a  candidate  process 
for  an  Action  Workout.  Once  the  process  is  identified,  more  analysis  is  required  to  seek 
out  the  critical  path  and  key  subtasks.  Process  owners,  management,  and  the 
headquarters  AWO  team  work  in  concert  to  determine  the  critical  path.  During  step  three 
of  the  AWO  process,  "Unit  Preparation  and  Data  Review,"  data  is  collected  from 
historical  archives,  observation,  and  analysis  of  videotapes  to  establish  baselines  for  the 
AWO  process  measures  (Action,  1997).  During  the  event,  AWO  team  members 
implement  time  and  motion  study  tools  to  formulate  process  improvements.  The 
"spaghetti  chart"  is  an  example  of  a  common  tool  used  by  AWO  team  members  to 
document  time  and  motion  of  a  particular  process  (Action,  1997).  The  spaghetti  chart  in 
Figure  1  illustrates  employee  movements  and  work  paths  for  a  particular  subtask  on  the 
B-1B  phase  inspection,  after  AWO  improvements  were  made.  To  facilitate  the 
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standardization  of  processes,  improvement  ideas  and  innovations  are  communicated 
throughout  the  Air  Force  so  that  similar  benefits  can  be  made  by  all. 


Figure  1.  Spaghetti  Chart,  B-1B  Phase  AWO  (Action,  1997) 

The  second  principle,  "Scientific  selection  of  workmen,"  suggests  that 
management  must  consciously  study  workers  and  employ  them  in  the  most  fitting  job 
possible.  Management  should  also  progressively  train  workers  in  their  duties  to  ensure 
optimal  performance  (Taylor,  1987:  72;  Thurston,  1998:  3). 
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Action  Workout  team  members  are  comprised  of  unit  personnel  who  have  varying 
degrees  of  expertise  in  the  identified  processes,  as  well  as  people  outside  of  the 
organization  who  are  totally  unfamiliar  with  the  process.  The  idea  is  to  get  a  mixture  of 
people  with  varying  backgrounds  to  provide  fresh,  new,  differing  points  of  view  and 
enhance  creativity.  Team  members  also  receive  training  on  AWO  concepts,  tools,  and 
approaches  to  improvement.  These  are  indirect  examples  of  the  "scientific  selection  of 
workers"  principle. 

The  third  principle  is  "Bringing  together  the  science  and  the  man"  (Taylor,  1987: 
73;  Thurston,  1998:  3).  In  other  words,  obtain  worker  cooperation  or  "buy-in"  of 
scientific  principles.  Positive  incentives,  such  as  better  treatment,  consideration  of  wants 
and  needs,  and  improved  communication  should  be  used  to  infuse  worker  cooperation,  as 
well  as  strict  negative  enforcement  of  principles.  In  addition,  an  environment  of  rapport 
and  trust  between  management  and  lower  level  workers  must  be  fostered.  Management, 
incidentally,  had  considerable  difficulty  coming  to  terms  with  this  new  concept. 

Quality  Air  Force  initiatives  in  general  reflect  characteristics  of  this  principle. 
Employee  buy-in  is  an  essential  element  to  any  change  initiative,  regardless  of  title. 

Unlike  scientific  management,  however.  Action  Workouts  empower  workers  and  utilize 
them  in  the  improvement  process.  Also,  by  relying  on  worker  inputs  for  improvement 
ideas,  Action  Workouts  cultivate  employee  buy-in  and  consequently  stand  a  better  chance 
for  success. 

The  final  principle,  "Division  of  work,"  was  considered  an  essential  characteristic 
of  scientific  management  (Taylor,  1987:  73-74;  Thurston,  1998:  3).  Taylor  argued  that  a 
50/50  split  in  work  between  management  and  worker  would  result  in  teamwork, 
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democracy,  and  cooperation.  Management  must  share  in  the  workload  by  determining 
the  best  possible  way  to  do  work  and  providing  the  worker  with  all  possible  means  to 
accomplish  their  job.  Work  must  be  analyzed  so  that  improved  methods  and  tools  are 
provided.  Adequate  training  and  equitable  feedback  should  also  be  given  to  ensure 
workers  reach  their  full  potential. 

To  fully  appreciate  this  concept  one  must  consider  the  history  of  manufacturing. 
Just  prior  to  the  industrial  revolution,  manufacturing  was  done  by  highly  skilled 
craftsmen  who  determined  what,  when,  where,  and  how  work  was  to  be  done  (Schroeder 
and  Robinson,  1991:  68).  Change,  brought  about  by  the  industrial  revolution  and  the 
advent  of  sophisticated  machinery,  diminished  the  demand  for  these  skilled  artisans,  and 
management  began  to  flourish.  Management  decided  how  work  was  to  be  done  and  told 
workers  what  to  do  and  not  to  think.  Current  management  philosophies  have  changed 
considerably  from  this  strict  authoritative  management  approach,  but  the  notions  of 
teamwork,  democracy,  and  cooperation  are  considered  mainstays.  The  focal  point  of 
both  Scientific  Management  and  the  Action  Workout  process  is  ultimately,  process 
improvement. 

Total  Quality  Management 

Total  Quality  Management  (TQM)  is  arguably  the  most  popular  approach  to 
organizational  improvement  today.  The  literature  surrounding  TQM  is  as  plentiful  as  the 
number  of  differing  interpretations  of  the  term.  In  order  to  establish  a  conceptual 
analysis  of  TQM,  it  may  be  helpful  to  start  with  a  definition  of  the  term.  From  this  a 
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review  of  underlying  principles  and  concepts  of  quality  will  be  made.  A  leading  textbook 

on  quality  defines  TQM  as  (Evans  and  Lindsay,  1996:  103): 

The  total,  company-wide  effort-through  full  involvement  of  the  entire  workforce 
and  a  focus  on  continuous  improvement-that  companies  use  to  achieve  customer 
satisfaction. 

The  goals  of  TQM  are  to  satisfy  the  customer,  prevent— not  correct  quality 
problems,  develop  an  attitude  of  continuous  improvement,  understand  how  to  effectively 
measure  performance  to  identify  opportunities  and  maintain  improvements,  and  eliminate 
sources  of  inefficiencies  and  costs  (Evans  and  Lindsay,  1996:  103).  The  basis  of  quality 
management  is  comprised  of  personal  prescriptions  and  philosophies  from  the  likes  of 
quality  "gurus"  such  as  Deming,  Juran,  Crosby,  Feigenbaum  and  Ishikawa  (Saraph  and 
others,  1989:  810-81 1;  Black  and  Porter,  1996:1-2).  W.  Edwards  Deming's  widely 
known  14  Points  for  Continuous  Improvement  perhaps  best  illustrates  the  TQM  concept. 
The  Deming  philosophy  centers  on  improving  product  and  service  quality  through 
variability  reductions  in  design  and  manufacturing  processes  (Evans  and  Lindsay,  1996: 
61).  At  the  time,  however,  the  14  points  were  regarded  by  some  as  ambiguous  and 
caused  confusion  among  businessmen  because  they  lacked  clear  rationales  (Evans  and 
Lindsay,  1996:  61).  Much  of  the  literature  surrounding  TQM  can  be  directly  or  indirectly 
linked  to  Deming's  points.  A  brief  listing  of  the  1 4  points  is  provided  in  Table  1 . 
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Table  1.  Deming's  14  points  (Deming,  1986:  23-96) 


1 .  Create  constancy  of  purpose  for  improvement  of  product  and  service 

2.  Adopt  the  new  philosophy 

3.  Cease  dependence  on  mass  inspection 

4.  End  the  practice  of  awarding  business  on  the  basis  of  price  tag  alone 

5.  Improve  constantly  and  forever  the  system  of  production  and  service 

6.  Institute  training 

7.  Adopt  and  institute  leadership 

8.  Drive  out  fear 

9.  Break  down  barriers  between  staff  areas 

10.  Eliminate  slogans,  exhortations,  and  targets  for  the  work  force 

1 1 .  Eliminate  quotas  for  the  work  force  and  numerical  goals  for  people  in  management 

12.  Remove  barriers  that  rob  people  of  pride  of  workmanship 

13.  Encourage  education  and  self-improvement  for  everyone 

14.  Take  action  to  accomplish  the  transformation 


Throughout  the  past  decade,  numerous  attempts  have  been  made  to  decipher  and 
integrate  the  critical  factors  of  TQM  from  the  myriad  papers  in  the  quality  literature.  In 
1989,  Saraph  and  others  conducted  a  thorough  review  and  synthesis  of  the  quality 
literature  to  develop  an  empirical  framework  based  on  eight  critical  factors  of  quality 
management  (Saraph  and  others,  1989:  810-11;  Black  and  Porter,  1996:  3-4).  Similarly, 
in  1992,  Steel  and  Jennings  identified  1 1  recurring  themes  in  the  TQM  literature  (Steel 
and  Jennings,  1992:  18-19).  In  this  article,  the  authors  designate  TQM  as  "a  family  of 
intervention  techniques"  (Steel  and  Jennings,  1992:18).  A  more  recent  study  done  by 


16 


Black  and  Porter  identified  ten  critical  factors  to  be  used  as  an  empirical  framework  of 
TQM  (Black  and  Porter,  1996:  3-5).  The  factors  are  compared  in  Table  2. 


Table  2.  Comparison  of  Total  Quality  Management  Factors 


Saraph's  8  Factor  Model 

Steel  and  Jennings' 

1 1  Elements  of  TQM 

Black  and  Porter's 

10  Critical  Factors  of  TQM 

Top  management 

commitment 

Top  management 

commitment 

Corporate  quality  culture 
Strategic  quality 
management 

Quality  data  and  reporting 

Measurement  bias 

Quality  improvement 
measurement  systems 

Training 

Problem-solving  training 

Employee  relations 

Employee  empowerment 

People  and  customer 
management 

Process  management 

Continuous  process 
improvement 

Operational  quality 
planning 

Product/service  design 

Quality  engineering 

External  interface 

management 

Supplier  quality 
management 

Supplier-relations 

management 

Supplier  partnerships 

Role  of  the  quality 
department 

Quality  policy  deployment 

Teamwork  structures 

Customer  relationship 
management 

Customer  satisfaction 

orientation 

Cross-functional  problem 
solving 

Communication  of 
improvement  information 

Changing  the  corporate  culture 
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As  depicted  in  Table  2,  similarities  exist  in  nearly  every  analysis.  Clearly,  as 
Black  and  Porter  reported,  the  major  differences  lie  in  the  absence  of  items  relating  to 
customer  satisfaction  and  customer  relationship  management  (Black  and  Porter,  1996: 
12).  Important  similarities  that  can  be  linked  to  the  Action  Workout  process  include  (1) 
top  management  commitment,  (2)  quality  measurement,  (3)  empowerment,  (4)  process 
improvement,  and  (5)  quality  policy  and  teamwork  structures. 

Another  familiar  and  commonly  used  source  of  TQM  factors  is  the  Malcolm 
Baldridge  National  Quality  Award  model.  The  annual  U.S.  National  Quality  Award  was 
established  when  President  Reagan  signed  Public  Law  100-107,  the  Malcolm  Baldridge 
National  Quality  Improvement  Act  of  1987.  Purposes  of  the  Award  are  "to  promote  an 
understanding  of  the  requirements  for  performance  excellence  and  competitiveness 
improvements  and  to  promote  sharing  of  information  on  successful  performance 
strategies"  (Malcolm,  1998).  Criteria  for  the  award  are  provided  for  business,  education, 
and  healthcare,  as  listed  in  Table  3. 


Table  3.  1998  Malcolm  Baldridge  National  Quality  Award  Criteria  (Malcolm,  1998) 


Business 

Education 

Medical 

Leadership 

Leadership 

Leadership 

Strategic  planning 

Strategic  planning 

Strategic  planning 

Customer  and  market  focus 

Student  and  stakeholder 

focus 

Focus  on  patients,  other 
customers,  and  markets 

Information  and  analysis 

Information  and  analysis 

Information  and  analysis 

Human  resource  focus 

Human  resource  focus 

Staff  focus 

Process  management 

Educational  and  support 
process  management 

Process  management 

Business  results 

School  performance  results 

Organizational  performance 
results 
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Quality.  Like  TQM,  the  quality  construct  is  difficult  to  define.  In  1984,  David 
A.  Garvin  conducted  a  study  to  synthesize  the  varying  definitions  of  product  quality.  He 
formulated  the  following  5  approaches  to  defining  quality:  (1)  transcendent,  (2)  product- 
based,  (3)  user-based,  (4)  manufacturing-based,  and  (5)  value-based  (Garvin,  1984:  25- 
28).  The  transcendent  approach  regards  quality  as  innate  excellence.  The  product-  based 
approach  defines  quality  as  a  measurable  level  associated  with  a  product  or  service 
attribute.  The  user-based  approach  focuses  on  customer  satisfaction.  The 
manufacturing-based  approach  equates  quality  to  meeting  specifications,  and  the  value- 
based  approach  measures  performance  and  price  (Garvin,  1984:  25-28). 

A  study  conducted  at  the  University  of  Scranton's  School  of  Management  used 
Garvin's  five  approaches  to  investigate  how  industry  managers  described  quality. 
Administrators  surveyed  a  group  of  managers  in  various  industries  to  help  answer  the 
evasive  quality  question.  Respondents  were  asked  to  match  their  definition  of  quality 
with  one  of  the  following  categories  cited  from  Garvin:  (1)  transcendent,  (2)  product- 
based,  (3)  user-based,  (4)  manufacturing-based,  and  (5)  value-based  (Sebastianelli  and 
Tamimi,  1996:34;  Garvin,  1984:  25-28).  Not  surprisingly,  results  of  the  study  indicated 
that,  for  the  most  part,  the  industry  in  which  the  respondents  were  employed  greatly 
influenced  their  choice  of  categories. 

According  to  A1  Hyde  of  the  Brookings  Institution,  the  cornerstones  of  quality  are 
process  measurement,  customer  feedback,  participative  management,  and  supplier 
cooperation  (Hyde,  1997:58).  Dr.  David  Chaudron  synopsizes  quality  in  terms  of  six 
elements;  focus  on  quality  and  prevention  of  problems,  cooperation  with  suppliers  and 
customers,  continuous  improvement  and  waste  elimination,  empowerment,  problem 
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solving/prevention,  and  using  measurements  to  back  decisions.  He  defines  quality  as 
"consistently  producing  what  the  customer  wants  while  reducing  errors  before  and  after 
delivery  to  the  customer,"  adding  that  quality  is  not  so  much  an  outcome  as  it  is  a  never- 
ending  process  of  continuous  improvement  (Chaudron,  1998).  The  Japanese  generally 
regard  quality  as  "anything  that  can  be  improved"  (Imai  1986:  xxiii). 

Armand  Feigenbaum,  renowned  quality  "guru"  and  author  of  the  1983  edition  of 
Total  Quality  Control,  has  a  different  perception  of  Japanese  and  American  quality.  In  an 
interview  with  Quality  Magazine  in  June  1995,  Feigenbaum  characterized  quality  as  an 
American  concept  that  U.S.  companies  have  followed  for  many  years.  "The  great  myth 
about  quality  is  that  it  was  paced  by  Japan,"  said  Feigenbaum,  adding,  "quality  is  an 
independent  concept  that  doesn't  necessarily  vary  from  industry  to  industry  and  country 
to  country"  (Litsikas,  1995:38).  Feigenbaum  contends  that  the  customer  defines  quality 
(Litsikas,  1995:38).  Feigenbaum's  wisdom  on  quality  was  so  irrefutable  that,  in  1994,  the 
American  Society  for  Quality  Control  adopted  his  interpretation  as  it's  official  definition 
of  quality:  "Quality  is  what  the  customer  says  it  is.  It’s  not  what  the  quality  professional 
says  it  is.  It’s  not  what  the  engineer  says  it  is.  It's  what  the  customer  says  it  is"  (Smith 
and  Whitehall,  1997:  42). 

By  now  it  should  be  apparent  that  there  is  no  single  definition  of  quality  or  TQM, 
but  rather  a  conglomeration  of  concepts,  criteria,  and  categories  that  can  be  adapted  to  fit 
different  scenarios.  Many  of  the  quality  factors  (e.g.,  management  commitment, 
measurement,  empowerment,  and  process  improvement)  that  appear  in  the  literature  must 
be  firmly  in  place  before  Action  Workout  improvement  efforts  can  even  be  attempted. 

The  fact  that  senior  leaders  at  Headquarters  Air  Combat  Command  have  established  and 


20 


fund  an  AWO  team  to  assist  units  at  the  wing  level  is  just  one  indication  of  top 
management  commitment.  Process  owners  and  commanders  at  the  wing  level  must  be 
committed  to  adopting  the  AWO  philosophy  and  fostering  an  environment  for  continuous 
improvement.  These  factors  form  the  foundation  from  which  continuous  process 
improvements  can  evolve. 

Empirical  TQM  Research.  Considering  the  commonality  and  stability  of  the 
quality  movement  in  the  world  today,  along  with  the  plethora  of  TQM  literature,  one 
would  assume  that  significant  empirical  evidence  exists  supporting  the  TQM  construct. 
This  is  not  the  case.  Empirical  research  surrounding  TQM  is  merely  just  beginning  to 
accumulate  (Steel  and  Jennings,  1992:  31;  Black  and  Porter,  1996:  1;  Hendricks  and 
Singhal,  1997:  1259;  Sebastianelli  and  Tamimi,  1998:  161).  Most  of  the  TQM  literature 
is  based  on  case  studies,  "anecdotal  evidence"  and,  as  previously  mentioned,  the  personal 
prescriptions  and  philosophies  of  quality  gurus  describing  quality  concepts  and  quality 
improvement  programs  (Saraph  and  others,  1989:  810;  Black  and  Porter,  1996:1-2; 

Davis,  1997:1 ;  Sebastianelli  and  Tamimi,  1998:  161).  Other  research  tends  to  focus  on 
TQM  barriers  and  unsuccessful  attempts  by  organizations  to  implement  TQM  (Ahire, 
1996:  18;  Beer  and  others,  1990:  158-161;  Davis  1997:  13-21;  Hyde,  1997:  62-63; 
Sebastianelli  and  Tamini,  1998:  161;  Sterman  and  others,  1997:  503-504).  Many  of  the 
barriers  or  reasons  for  unsuccessful  TQM  efforts  are  (as  reported  by  Dr.  Ahire):  "lack  of 
top  management  commitment,  unrealistic  expectations  about  time  and  cost  of 
implementation,  over-  or  under-reliance  on  statistical  methods,  and  failure  to  develop  and 
sustain  a  quality  culture"  (Ahire,  1996:  18).  Despite  initial  skepticism,  the  TQM 
philosophy  seems  to  be  ingrained  throughout  both  the  public  and  private  sectors.  A 
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review  of  empirical  research  linking  quality  initiatives  with  improved  organizational 
performance  is  provided  in  the  following  paragraphs. 

Research  investigating  the  effects  of  Quality  Circles  (QC),  which  are  small  groups 
of  workers  from  the  same  work  center  who  periodically  meet  to  discuss,  study,  and 
analyze  work-related  problems  (Steel  and  others,  1990:  365)  has  produced  mixed  results. 
For  example,  in  1985,  Steel  and  others  conducted  a  longitudinal  study  to  evaluate  the 
success  of  QC  programs  at  two  U.S.  Army  units.  Results  of  the  study  showed  significant 
QC  benefits  for  one  of  the  units,  while  no  significant  gains  were  identified  at  the  second 
unit  (Steel  and  others,  1985:  99).  Dr.  Steel,  and  others,  conducted  a  similar  study  in 
1990.  The  longitudinal  study  evaluated  the  effects  of  QC's  on  labor-relations  criteria  in  a 
division  of  the  United  States  Department  of  the  Treasury.  Results  of  the  study  showed 
that  QC's  at  the  federal  mint  had  minimal  impact  on  key  organizational  outcomes.  The 
study  did  find  evidence  of  QC  gains  on  other  evaluation  criteria  (Steel,  and  others,  1990: 
379). 

Richard  J.  Magjuka  developed  a  survey  to  examine  how  manufacturers  design  and 
administer  employee  involvement  processes  to  effectively  contribute  to  continuous 
improvement  objectives.  The  survey  was  distributed  among  1,000  firms  affiliated  with 
the  Association  for  Quality  and  Participation,  a  non-profit  association  that  promotes 
employee  involvement  programs  and  total  quality  improvement  in  U.S.  corporations 
(Magjuka,  1994:  621).  Four  key  findings  of  the  study  suggest  that  quality  performance  is 
enhanced  when:  (1)  teams  operate  under  open  and  unrestricted  information  access 
structures,  (2)  employees  are  required  to  participate  in  TQM  efforts,  (3)  teams  are 
permanent  (as  opposed  to  temporary)  task  groups,  and  (4)  teams  operate  within  a  goal- 
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setting  system  (Magjuka,  1994:  626).  The  study  also  found  that  continuous  improvement 
TQM  programs  were  still  regarded  as  relatively  new  initiatives  in  many  of  the  firms 
surveyed  (Magjuka,  1994:  626). 

In  1991,  the  United  States  General  Accounting  Office  (GAO)  accomplished  one 
of  the  more  frequently  cited  empirical  studies  linking  TQM  with  improved  performance. 
The  GAO  study  was  a  review  of  the  top  20  applicants  for  the  1988  and  1989  Malcolm 
Baldridge  National  Quality  Award.  The  report  indicated  that  "Companies  that  adopted 
quality  management  practices  experienced  an  overall  improvement  in  corporate 
performance"  (GAO,  1991 :2).  The  study  focused  on  and  identified  improvements  in  the 
areas  of  employee  relations,  operating  procedures,  customer  satisfaction,  and  financial 
performance  (GAO,  1991 :  6),  and  provided  robust  empirical  evidence  supporting  the 
implementation  of  TQM  (Sterman  et  al.,  1997:  503-4;  Black  and  Porter,  1996:2). 

Thomas  J.  Douglas  reported  similar  findings  in  his  dissertation  at  the  University 
of  Tennessee  in  1997.  Douglas  developed  and  used  a  resource-based  perspective  model 
to  study  the  relationship  between  TQM  and  organizational  performance  in  the  medical 
services  industry,  surveying  over  500  hospitals  in  19  metropolitan  areas  across  the  United 
States.  Regression  analysis  of  the  193  responding  hospitals  indicated  that  the 
implementation  of  TQM  was  positively  related  to  perceived  financial  performance, 
archival  return  on  assets,  and  quality  of  service  (Douglas,  1997:2). 

Thomas  C.  Powell  conducted  a  review  and  empirical  study  of  TQM,  and  it's 
effects  on  competitive  advantage  in  1995.  Despite  initial  criticisms  and  reported 
shortcomings  associated  with  instituting  TQM,  such  as  excessive  increases  in  cost, 
paperwork,  and  time,  Powell  labeled  TQM  as  an  "irrepressible,  globally  pervasive 
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strategic  force  in  today's  industrial  economy"  (Powell,  1995:  16).  He  also  referred  to  the 
literature  encompassing  the  overall  effectiveness  of  TQM  as  "unclear  and  under¬ 
examined"  (Powell,  1995:  16).  Results  of  the  study  showed  evidence  linking  TQM  with 
improved  operational  performance.  However,  unlike  previously  cited  studies,  Powell 
attributed  the  improved  performance  outcomes  to  "tacit  resources,"  or  implied  behavioral 
features  of  TQM,  such  as  "open  culture,  employee  empowerment,  and  executive 
commitment"  (Powell,  1995:  15).  Furthermore,  Powell  indicated  that  many  of  the  TQM 
tools  and  techniques,  such  as  quality  training,  process  improvement,  and  benchmarking, 
do  not  lead  to  enhanced  operational  performance,  or  specifically,  competitive  advantage 
(Powell,  1995:  15).  The  author  concluded  that  organizations  that  acquire  these  tacit 
resources  can  "outperform  competitors  with  or  without  the  accompanying  TQM 
ideology"  (Powell,  1995:  15). 

Hendricks  and  Singhal  performed  a  more  recent  study  linking  the  implementation 
of  TQM  with  operational  performance  in  1997.  Using  quality  award  winning  firms  as  the 
basis  for  the  study,  Hendricks  and  Singhal  were  able  to  provide  "strong  evidence  that 
firms  that  have  won  quality  awards  outperform  a  control  sample  on  operating  income- 
based  measures"  (Hendricks  and  Singhal,  1997:  1271).  Another  important  finding  of  the 
study  indicated  that  early  benefits  of  TQM  programs  might  outweigh  implementation 
costs  (Hendricks  and  Singhal,  1997:  1271). 

Continuous  Improvement 

While  there  is  no  consensus  definition  of  TQM  among  the  "quality  gurus,"  the 
notion  of  continuous  improvement  is  prevalent  in  almost  all  of  the  TQM  literature. 
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Continuous  improvement  is  the  heart  of  TQM— one  of  the  single-most  important  elements 
of  any  quality  initiative.  Deming  made  a  clear  distinction  between  improvement 
approaches  in  the  East  and  West  (Japanese  and  US  industry)  in  his  book,  Out  of  Crisis. 

He  contended  that  Japanese  companies  favor  the  "gradualist  approach"  to  improvement, 
while  Western  companies  favor  the  "great-leap"  approach  (Deming,  1996:  23-41).  The 
two  approaches  can  be  viewed  as  KAIZEN  (gradual),  and  innovation  (great-leap).  In 
Japan,  improvement  is  defined  as  KAIZEN  and  innovation  (Imai,  1986:  xx).  KAIZEN  is 
instilled  as  "a  mindset  inextricably  linked  to  maintaining  and  improving  standards." 
KAIZEN  strategy  maintains  and  improves  standards  by  implementing  minute,  gradual 
improvements.  Innovation,  on  the  other  hand,  brings  about  radical  improvements  as  a 
result  of  technological  advancements  or  equipment  investments  (Imai,  1986:  xx). 

Deming  similarly  described  KAIZEN  as  undramatic,  subtle,  continuous  improvement 
that  could  be  characterized  as  people-oriented.  He  described  innovation  as  a  dramatic, 
one-shot  phenomenon  characterized  as  technology  and  money-oriented  (Deming,  1996: 
23-41).  From  this  it  can  be  said  that  a  successful  continuous  improvement  effort  must 
exhibit  both  KAIZEN  and  innovation  approaches. 

Quality  Air  Force  attempts  to  instill  both  approaches  in  the  pursuit  of  continuous 
improvement.  For  example,  Air  Force  policy  dictates  that  commanders  follow  specific 
operating  styles  to  foster  an  environment  conducive  to  continuous  improvement.  These 
operating  styles  can  be  viewed  as  KAIZEN  strategy  (Bemowski,  1994:  26): 

•  Create  a  working  climate  that  inspires  trust,  teamwork,  and  pride. 

•  Delegate  responsibility  and  authority,  and  accept  accountability  for  results. 

•  Set  goals,  measure  progress,  and  reward  performance. 

•  Give  everyone  a  stake  in  the  outcome. 
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Action  workouts,  by  definition,  are  employed  to  bring  about  "dramatic  productivity 
improvements  in  any  organization"  (DoD,  1996b:  41-45).  Obviously,  an  AWO  event  can 
be  considered  as  an  innovation  approach  to  improvement. 

Successful  Continuous  Improvement.  Much  of  the  continuous  improvement 
literature  emphasizes  the  importance  of  rank  and  file  worker  involvement  in 
improvement  efforts.  (Suzaki,  1993:  87-116;  Imai,  1996:  217;  Choi  and  others,  1997: 

45).  Action  Workouts  epitomize  this  concept  by  relying  on  worker  inputs  during  the 
AWO  process.  Process  owners  attack  waste  and  create  innovative  ideas  while  being 
assisted  by  AWO  team  leaders,  headquarters  functional  policy  experts,  engineers,  and 
technical  order  specialists.  The  supporting  cast  is  available  to  provide  guidance,  clear 
obstacles,  and  make  on  the  spot  changes  to  written  procedures  (AWO,  1 997).  Worker 
involvement,  however,  is  not  the  only  contributing  factor  for  successful  improvement 
initiatives.  Schroeder  and  Robinson  (1991)  proposed  four  fundamental  principles  for 
successful  continuous  improvement  initiatives  (Shroeder  and  Robinson,  1991:  74-79). 
First,  "It  must  be  clearly  understood  that  improvements  at  first  cause  dislocation,  and 
almost  always  require  time  before  they  prove  worthwhile. "  In  other  words,  new 
procedures  brought  about  by  continuous  improvement  programs  are  generally  more 
difficult  and  time-consuming  than  the  familiar  "old"  procedures.  For  this  reason, 
improvements  require  a  certain  amount  of  time  or  learning  period  before  true  effects  can 
be  realized.  Second,  "Operating  practices  that  restrict  the  flow  of  ideas  must  be 
eliminated. "  Organizational  culture  should  encourage  close  relationships  between 
management  and  workers  and  eliminate  barriers  to  communication.  Employment 
guarantees  and  incentive  programs  also  help  to  encourage  creative  thinking  and  idea 
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generation.  Third,  "Employees  must  be  continuously  trained  and  developed,  particularly 
in  techniques  of  methods  improvement. "  Employee  development  is  more  than  teaching 
people  how  to  do  their  job.  Problem  solving  techniques  must  be  taught  so  employees  can 
improve  the  methods  by  which  their  work  is  done.  Finally,  "A  continuous  improvement 
effort  needs  an  efficient  mechanism  to  handle  improvement  ideas. "  A  continuous 
improvement  program  must  have  well-planned  means  to  gather,  evaluate,  implement,  and 
reward  improvement  ideas.  Effective  suggestion  programs  should  exhibit  the  following 
characteristics.  They  should  be  simple  to  participate  in,  and  encouraged.  Competent 
parties,  who  should  provide  employees  rapid  feedback,  should  review  all  ideas. 

Approved  ideas  should  be  implemented  as  quickly  as  possible. 

These  principles  obviously  relate  to  the  Action  Workout  improvement  process, 
and  seem  to  be  well  established  in  the  Air  Force's  quality  culture  today.  The  true  test  will 
be  the  sustainment  of  continuous  improvement  initiatives  through  the  long-term. 

Change 

With  almost  every  improvement  there  exists  change.  Not  every  change,  however,  results 
in  improvement  (Provost  and  Langley,  1998:  31,  Provost  and  Sproul,  1996:  101).  A 
recent  study  conducted  by  Coopers  and  Lybrand  Consulting,  in  conjunction  with  Opinion 
Research  International,  identified  the  following  five  key  elements  to  successfully 
initiating  change  in  the  workplace:  (1)  strong  leadership,  (2)  effective  communication,  (3) 
a  tight  alignment  of  people  and  organizations,  (4)  adequate  training  and  funding,  and  (5) 
clear  definition  of  compelling  reasons  to  change  (Smith,  1998:  45-48).  These  key 
elements  apply  to  both  large-scale  (organizational-level)  and  small-scale  (transactional- 
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level)  changes.  Other  literature  suggests  full  and  open  communication,  building  trust, 
and  encouraging  a  culture  of  change  (Barrier,  1998:  31-34;  Suzaki,  1983: 16-40). 

The  principles  of  effective  change  are  similar  to  the  principles  of  continuous 
improvement  and  are  equally  important  to  the  overall  success  of  Action  Workouts  in  the 
Air  Force  today. 

Summary 

The  literature  surrounding  Scientific  Management,  Total  Quality  Management, 
and  Continuous  Improvement  were  explored  to  provide  a  basis  for  this  research  effort. 
While  many  of  the  concepts  discussed  are  difficult  to  grasp  absolutely,  the  major 
principles  can  be  directly  or  indirectly  associated  with  the  Action  Workout  process.  The 
following  chapter  will  explain  the  method  used  in  this  study. 
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III.  Method 


Overview 

The  objective  of  this  study  is  to  gather  and  analyze  aircraft  inspection  data  to 
determine  if  quality  enhancements  are  realized  after  an  action  workout  is  accomplished. 
In  order  to  achieve  this  objective,  an  appropriate  research  design  and  data  analysis 
method  were  required.  This  chapter  will  discuss  the  method  used  to  achieve  the  research 
objective. 

Sampling  Design 

The  target  population  of  interest  to  this  study  consists  of  all  Air  Combat 
Command  units  that  have  had  an  Action  Workout  accomplished  on  one  or  more  critical 
processes.  At  the  beginning  of  this  research  effort,  twenty  processes  were  identified  as 
having  been  subjected  to  the  A  WO  improvement  method.  Eight  of  the  identified  twenty 
AWO  events  were  accomplished  on  major  aircraft  inspection  processes.  These  processes 
were  chosen  as  the  target  population  for  a  variety  of  reasons.  First,  a  major  aircraft 
inspection  process  is  crucial  to  operational  readiness  and  to  the  mission  of  the  Air  Force. 
Second,  the  process  is  relatively  complex  and  provided  a  multitude  of  improvement 
opportunities.  Finally,  the  necessary  data  or  process  measurements  were  obtainable  and 
provided  for  useful  analysis. 

Research  Design 

The  ex  post  facto  research  design,  utilizing  pretest  and  post-test  data,  was  chosen 
as  the  most  feasible  approach  for  this  study.  The  scope  of  this  research,  along  with  budget 
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and  time  constraints,  made  it  nearly  impossible  to  conduct  an  experimental  study  that 
would  provide  a  more  powerful  evaluation  of  the  AWO  process.  Although  manipulation 
of  variables  could  not  be  made,  the  ex  post  facto  design,  when  used  with  caution,  can  still 
provide  sufficient  evidence  of  causation  (Cooper  and  Emory,  1995:  142).  While  the 
focus  of  the  study  is  to  determine  whether  a  causal  relationship  exists  between  Action 
Workout's  and  the  overall  quality  of  the  processes  studied,  it  is  imperative  to  note  that 
multiple  causes  may,  and  most  likely  do,  exist.  These  causes  or  extraneous  variables  will 
be  addressed  in  the  following  chapters. 

To  ensure  that  an  adequate  sample  size  could  be  obtained,  careful  consideration 
was  given  to  the  timing  of  each  AWO  event.  Since  the  frequency  of  inspections  varied 
according  to  weapon  system,  it  was  determined  that  approximately  two  years  of  data 
would  be  sufficient.  The  goal  was  to  obtain  12  months  of  data  both  before  and  after  the 
date  of  the  AWO  event,  so  that  comparisons  could  be  made.  Three  units  were  selected 
that  closely  met  these  criteria: 

1.  55  WG,  Offutt  AFB,  NE.  C-135  Isochronal  Inspection  Process 

2.  552  ACW,  Tinker  AFB,  OK.  E-3  Isochronal  Inspection  Process 

3.  355  WG,  Davis  Monthan  AFB,  A Z.  A-10  Phase  Inspection  Process 

Measures 

The  data  used  in  this  study  were  archival  process  measures,  or  more  specifically, 
Quality  Verification  Assessment  results  formulated  by  Quality  Assurance  personnel.  As 
previously  mentioned,  a  Quality  Verification  Assessment  is  a  follow-up 
inspection/assessment  of  an  item  (equipment)  or  process  (maintenance  or  repair  action) 
performed  to  determine  "an  estimate  of  quality,  relative  to  a  predetermined  level  of 
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expected  or  acceptable  status"  (Do  A,  1997:  135).  The  measure  of  whether  an  assessment 
was  acceptable  or  unacceptable  is  described  as  a  Quality  Verification  Result  (QVR).  The 
QVR  is  based  on  Acceptable  Quality  Levels  established  by  Quality  Assurance  personnel. 
There  are  three  QVR  categories:  A  QVR  1  indicates  that  the  Acceptable  Quality  Level 
was  met,  a  QVR  2  indicates  that  the  Acceptable  Quality  Level  was  not  met,  and  a  QVR  3 
indicates  that  the  Acceptable  Quality  Level  was  not  met  due  to  a  safety  or  serious  (major) 
defect  (DoA,  1997:  138).  A  brief  explanation  of  Quality  Assurance  is  given  to  provide  a 
better  understanding  of  the  data  source. 

Quality  Assurance.  Quality  Assurance  can  be  described  as  a  separate  unit 
comprised  of  a  variety  of  personnel  with  differing  maintenance  specialties.  The  role  of 
Quality  Assurance,  according  to  ACCI  21-101,  is  to  assess,  analyze,  and  identify 
problems  in  Wing  maintenance  areas;  assess  the  quality  of  training;  determine  aircraft 
and  equipment  condition;  and  increase  reliability  and  maintainability.  Quality 
Verification  Assessments  are  only  one  of  many  elements  that  make  up  Quality  Assurance 
Program's,  which  are  designed  to  improve  combat  capability  through  high  quality 
maintenance  and  effective  maintenance  training  (DoA,  1997:  127).  According  to  ACCI 
21-101,  Quality  Assurance  is  required  to  perform  a  Quality  Verification  Assessment 
(follow-up)  of  every  major  aircraft  inspection  to  verify  that  the  inspection  was  properly 
accomplished  (DoA,  1997:  128).  All  discrepancies  noted  during  the  Quality  Verification 
Assessment  are  documented,  and  entered  into  the  Quality  Assurance  database. 
Discrepancies  are  classified  as  either  major  or  minor  defects.  Defects  that  may  lead  to 
severe  damage  to  equipment  or  injury  to  personnel  are  classified  as  major  defects.  A 
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minor  defect  is  a  missed  or  incomplete  inspection  item  that  will  not  result  in  excessive 
damage  to  equipment  or  safety  of  flight. 

To  obtain  a  representative  sample  of  inspection  results,  Quality  Verification 
Assessment  data  was  collected  from  three  separate  Quality  Assurance  inspection  teams 
who  conduct  maintenance  assessments/inspections  on  the  A- 10,  C-135,  and  E-3  weapon 
systems  at  the  355  WG,  Davis  Monthan  AFB,  A Z.,  the  55  WG,  Offutt  AFB,  NE.,  and  the 
552  ACW,  Tinker  AFB,  OK.,  respectively.  The  three  teams  were  chosen  primarily 
because  of  the  availability  of  data  and  represent  a  wide  array  of  inspection  requirements 
and  improvement  opportunities. 

55  WG,  Offutt  AFB,  NE.  C-135  Isochronal  Inspection  Process.  The  Quality 
Assurance  databases  provided  QVR  ratings,  overall  Quality  Assurance  scores,  and  total 
number  of  inspection  days  per  aircraft.  Acceptable  Quality  Levels  established  the  QVR 
ratings.  Adding  the  points  associated  with  minor  and  major  defects  noted  during  the 
assessment  derived  overall  scores.  In  all  cases,  one  point  is  given  for  each  minor  defect, 
and  four  points  for  each  major  defect.  For  example,  a  Quality  Verification  Assessment 
that  identified  two  minor  defects,  and  one  major  defect,  would  have  an  overall  score 
equal  to  six.  Data  was  collected  on  82  Quality  Verification  Assessment's  conducted 
between  February  1996,  and  October  1997.  The  data  was  then  split  to  form  the  pretest 
and  post-test  data  sets.  The  number  of  assessments  accomplished  before  the  AWO  was 
23,  while  the  number  accomplished  after  the  event  was  59. 

552  ACW,  Tinker  AFB,  OK.  E-3  Isochronal  Inspection  Process.  In  addition 
to  QVR  ratings  and  overall  scores,  the  552nd  Quality  Assurance  databases  provided  the 
number  of  major  and  minor  defects  that  were  found  during  each  assessment.  Data 
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reflecting  total  inspection  days  per  aircraft  could  not  be  obtained.  The  total  sample  size 
was  75.  The  data  was  then  split  to  form  pretest  and  post-test  data  sets.  The  number  of 
assessment's  accomplished  before  the  AWO  event  was  9,  while  the  number  accomplished 
after  the  event  was  66. 

355  WG,  Davis  Monthan  AFB,  AZ.  A-10  Phase  Inspection  Process.  The 

355th  Quality  Assurance  databases  contained  information  similar  to  that  of  the  552nd 
Quality  Assurance.  Data  reflecting  total  inspection  days  per  aircraft  could  not  be 
obtained.  The  total  sample  size  was  86.  The  data  was  also  split  to  form  pretest  and  post¬ 
test  data  sets.  Twenty-four  assessments  were  accomplished  before  the  AWO  event,  and 
sixty-two  after. 

Once  again,  it  is  important  to  note  that  data  analyzed  for  this  study  was  obtained 
from  Quality  Verification  Assessments,  conducted  by  Quality  Assurance  personnel. 
Maintenance  personnel  assigned  to  their  respective  Inspection  Sections  accomplished  the 
actual  Phase  and  Isochronal  inspections. 

Procedure 

Archival  assessment  data  were  obtained  from  databases  maintained  by  each  unit. 
In  all  three  cases,  the  amount  of  useful  data  reflecting  process  measures  prior  to  the  AWO 
interventions  (henceforth  the  pretest  scores)  were  fewer  than  the  data  reflecting  process 
measures  after  the  AWO  interventions  (henceforth  the  post-test  scores).  This  anomaly 
can  best  be  attributed  to  advances  or  changes  in  the  information  management  systems 
used  by  the  units.  Changes  in  maintenance  inspection  concepts  (phase  to  isochronal)  also 
contributed  to  these  shortfalls,  particularly  at  Tinker  AFB,  OK.  In  addition,  the  type  of 
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information,  as  well  as  the  level  of  detail,  varied  between  Quality  Assurance  databases  at 
the  three  selected  units  (55  WG,  355  WG,  and  552  ACW).  For  example,  the  55  WG  was 
the  only  unit  with  inspection  time  data  and  the  only  unit  not  to  have  data  pertaining  to  the 
number  of  major  and  minor  defects.  Because  the  data  were  not  aggregated  across 
samples,  metric  differences  were  not  an  analytic  problem. 

Data  Analysis 

In  order  to  improve  the  reliability  of  the  study,  pretest  and  post-test  assessment 
data  from  similar  processes  performed  at  three  separate  units  were  analyzed.  Descriptive 
statistics  were  used  to  identify  means,  variances,  and  standard  deviations.  Parametric 
t-tests  were  then  used  to  determine  the  significance  of  any  differences  between  the 
sample  means  (Quality  Assurance  scores,  majors,  and  minors),  as  applicable.  Chi-square 
analysis  was  used  to  determine  differences  between  QVR  ratings.  The  data  are  free  from 
selection  bias  and  independent. 
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IV.  Results 


Overview 

This  chapter  reports  findings  and  differences  between  pretest  and  post-test  data 
samples  for  three  separate  units.  A  brief  synopsis  of  each  unit's  Action  Workout  trip 
report  is  provided  as  supplemental  information  only.  Action  Workout  trip  reports  for  the 
units  identified  in  this  study,  as  well  as  every  Air  Combat  Command  unit  that  hosted  an 
event,  can  be  obtained  from  the  Air  Combat  Command  Quality  and  Management 
Innovation  Squadron's  internet  homepage  (Action,  1997).  The  independent  results  are 
provided  in  the  following  three  sections:  Offutt  Results,  Davis-Monthan  Results,  and 
Tinker  Results. 

Offutt  Results 

Offutt  AFB  pretest  and  post-test  data  sets  reflecting  QVR  ratings,  mean  Quality 
Assurance  scores,  and  total  number  of  inspection  days  per  aircraft  were  analyzed. 

AWO  Summary.  The  Action  Workout  effort  on  the  C-135  Isochronal  inspection 
process  was  accomplished  during  the  week  of  29  July  through  2  August  1996.  Due  to  the 
limited  number  of  mission-critical  aircraft,  a  requirement  existed  to  return  aircraft  to  fully 
mission  capable  status  as  quickly  as  possible.  The  urgent  need  for  improvement  stemmed 
from  the  unacceptable  number  of  days  required  to  perform  an  inspection-approximately 
eight  days  per  aircraft.  To  complicate  matters,  maintenance  personnel  assigned  to  the 
Inspection  Section  were  responsible  for  performing  inspections  on  13  different  models  of 
C-135  aircraft.  Five  teams  were  assembled  to  focus  on  critical  paths  identified  during  the 
AWO  team  visit  (pre-work).  Each  team  implemented  numerous  innovations  and 
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improvement  ideas  that  resulted  in  impressive  reductions  in  all  target  areas:  set-up  time, 
man-hours,  cycle-time,  and  distance  traveled.  For  example,  the  wing  inspection  team 
reported  a  72%  reduction  in  cycle  time  and  an  89  %  reduction  in  distance  traveled  for  the 
wing  inspection  portion  of  the  process.  The  lubrication  team  reported  a  61%  reduction  in 
cycle-time  and  man-hours  required  to  lubricate  an  aircraft,  as  well  as  an  88%  reduction  in 
distance  traveled.  Eliminating,  streamlining  and  changing  technical  order  inspection 
requirements  brought  about  many  of  the  improvement  results.  Other  innovations,  such  as 
the  repositioning  of  tools  and  equipment,  designing  of  tool  belts  and  parts  bags,  and 
acquisition  of  special  equipment,  also  contributed  to  the  overall  savings.  In  the  end,  the 
entire  C-135  isochronal  inspection  process  was  reportedly  reduced  to  an  average  of  "just 
over  6  days"  (Action,  1997). 


Table  4.  Offutt  AFB  QVR  Ratings 


Pretest3 

Post-testb 

X2 

Variables 

Freq 

% 

Freq 

% 

(df) 

QVR  1 

13 

56.50 

49 

83.10 

6.32* 

1 

QVR  3 

10 

43.50 

10 

16.90 

aN=  23 
b  N  =  59 
*  p  <  .05 


QVR  Ratings.  Table  4  illustrates  the  frequency  and  percentage  of  both  QVR  1 
and  QVR  3  ratings  that  complete  the  pretest  and  post-test  data  sets  at  Offutt  AFB.  Pretest 
data  indicated  that  13  of  23  inspections  (56.5%)  received  QVR  1  ratings  and  10 
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inspections  (43.5%)  received  QVR  3  ratings.  Post-test  data  showed  49  QVR  1  ratings 
(83.1%)  and  10  QVR  3  ratings  (16.9%).  There  were  no  QVR  2  ratings  in  either  data  set. 
Results  indicate  a  26.6%  increase  in  the  number  of  QVR  1  ratings  received  after  the 
AWO  intervention.  Nonparametric  chi-square  analysis  indicated  the  difference  between 
pretest  and  post-test  QVR  ratings  was  statistically  significant  (X2  =  6.3156,  df=  1).  The 
QVR  rating,  otherwise  referred  to  as  the  "pass/fail  rating,"  is  the  primary,  bottom  line 
performance  measure  used  in  many  aircraft  maintenance  processes  today.  Other 
performance  measures  used  in  this  study,  such  as  score,  major,  and  minor  defects, 
ultimately  contribute  to  the  QVR  rating,  but  do  not  receive  the  same  amount  of  scrutiny. 
In  terms  of  process  improvement,  the  26.6%  increase  in  QVR  1  ratings  realized  at  Offutt 
AFB  is  substantial.  The  following  table  illustrates  pretest  and  post-test  mean  Quality 
Assurance  scores. 


Table  5.  Offutt  AFB  Quality  Assurance  Scores 


Pretest3 

Post-test6 

Variable 

Mean 

S.D. 

Variance 

Mean 

S.D.  Variance  T-stat 

Score 

12.87 

5.70 

32.48 

9.86 

6.32  39.91  *-1.99 

a  N  =  23 
bN  =  59 
*  p  <  .05 


Quality  Assurance  Scores.  Recall  from  the  previous  chapter  that  overall  Quality 
Assurance  scores  are  derived  by  adding  the  points  associated  with  the  number  of  major 
(four  points)  and  minor  (one  point)  defects  noted  during  the  Quality  Verification 
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Assessment.  Data  analysis  indicates  that,  on  average,  fewer  discrepancies  are  being 
found  during  follow-up  assessments  at  Offutt  AFB  since  the  AWO  intervention.  The 
mean  score  for  the  pretest  data  set  was  12.9.  The  mean  score  for  the  post-test  data  set 
was  9.9.  Results  indicate  a  23.3%  reduction  (3  points)  in  overall  score.  T-test  analysis 
for  two  independent  samples  confirmed  the  difference  between  mean  scores  was 
statistically  significant  (p=.0504).  Considering  the  complexity  of  the  C-135  isochronal 
inspection  process,  a  three-point  reduction  is  noteworthy. 


Table  6.  Offutt  AFB  Inpsection  Days 


Pretest3 

Post-testb 

Variable 

Mean 

S.D. 

Variance 

Mean 

S.D.  Variance 

T-stat 

Days 

7.65 

1.07 

1.15 

7.76 

1.30  1.70 

0.36 

aN  =  23 
bN  =  59 


Inspection  Days.  One  of  the  primary  objectives  of  an  Action  Workout  is  to  cut 
process  cycle  times.  As  mentioned  above,  the  AWO  team  reported  an  expected  1 .5-day 
reduction  per  isochronal  inspection.  These  estimates,  however,  were  based  on  a  single 
event.  Statistical  analysis  of  the  pretest  and  post-test  data  sets  revealed  that  there  was  no 
significant  difference  between  pretest  and  post-test  inspection  times  (see  Table  6).  The 
average  number  of  days  required  to  perform  an  inspection  prior  to  the  AWO  intervention 
was  7.7,  compared  to  7.8  days  required  after  the  intervention. 
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Davis-Monthan  Results 


Pretest  and  post-test  data  sets  for  Davis  Monthan  AFB  reflect  mean  QVR  ratings, 
Quality  Assurance  scores,  and  number  of  major  and  minor  defects  noted  per  inspection. 

AWO  Summary.  The  Action  Workout  on  the  A- 10  phase  inspection  process 
took  place  between  19  May  and  23  May  1997.  Five  teams  were  also  formed  to  narrow 
the  focus  of  the  event.  Substantial  improvements  were  reported  by  eliminating 
unnecessary  process  steps,  consolidating  equipment,  reducing  travel  distance,  and 
incorporating  a  revised  inspection  process  and  phase  schedule.  Some  notable 
improvement  results  include  cycle  time  and  man-hour  savings  of  41%,  for  portions  of  the 
"look  phase"  inspection,  as  well  as  a  65%  reduction  in  distance  traveled.  The  structural 
repair  team  reported  an  82%  reduction  in  cycle-time  and  man-hours  required  for 
corrosion  inspections.  The  total  process  cycle  time  for  the  A- 10  phase  inspection  process 
reportedly  dropped  from  75  to  48  hours,  a  36%  decrease  (Action,  1997). 


Table  7.  Davis  Monthan  AFB  QVR  Ratings 


Pretest3 

Post-testb 

X2 

Variables 

Freq 

% 

Freq 

% 

(df) 

QVR  1 

14 

58.30 

34 

54.80 

0.09 

1 

QVR  3 

10 

41.70 

28 

45.20 

aN  =24 
bN=6  2 
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QVR  Ratings.  Table  7  illustrates  Davis  Monthan  AFB  QVR  ratings.  Pretest 
data  indicated  that  14  of  24  inspections  received  a  QVR  1  rating  (58.3%),  and  10 
inspections  received  QVR  3  ratings  (41.7%).  Post-test  data  showed  34  QVR  1  ratings 
(54.8%)  and  28  QVR  3  ratings  (45.2%).  There  were  no  QVR  2  ratings  in  either  data  set. 
Results  indicate  a  3.5%  decrease  in  the  number  of  QVR  1  ratings  received  after  the  AWO 
intervention.  This  slight  decrease,  however,  was  not  found  to  be  significant. 


Table  8.  Davis  Monthan  AFB  Quality  Assurance  Scores 


Pretest3 

Post-testb 

Variable 

Mean 

S.D. 

Variance 

Mean 

S.D.  Variance 

T-stat 

Score 

5.29 

5.92 

35.00 

6.69 

6.04  36.48 

-0.98 

aV  =  24 
bN  =  62 


Quality  Assurance  Scores.  Table  8  shows  pretest  and  post-test  Quality 
Assurance  scores  for  Davis  Monthan  AFB.  The  mean  score  for  the  pretest  data  set  was 
5.3.  The  mean  score  for  the  post-test  data  set  was  6.7.  Results  indicate  a  20.9%  increase 
in  overall  score.  T-test  analysis  for  two  independent  samples  indicated  that  the  difference 
between  mean  scores  was  not  statistically  significant  (p=. 3326).  Table  9  shows  the 
pretest  and  post-test  data  for  major  and  minor  defects. 
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Table  9.  Davis  Monthan  AFB  Major  and  Minor  Defects 


Pretest3 

Post-testb 

Variable 

Mean 

S.D. 

Variance 

Mean 

S.D. 

Variance 

T-stat 

Major 

0.67 

1.67 

1.36 

0.63 

1.01 

1.02 

0.15 

Minor 

2.63 

2.20 

4.85 

4.18 

3.66 

13.39 

*  -2.40 

aN  =24 
bN  =  62 
*  p  <  .05 


Major  Defects.  Virtually  no  significant  difference  was  found  between  the  mean 
number  of  major  defects  in  the  pretest  and  post-test  data  sets.  As  Table  9  indicates,  the 
average  number  of  major  defects  found  per  inspection  prior  to  the  A  WO  intervention  was 
.7,  compared  to  .6  after  the  AWO  intervention. 

Minor  Defects.  The  difference  between  the  mean  number  of  minor  defects  in 
the  pretest  and  post-test  data  sets,  however,  was  statistically  significant.  The  average 
number  of  minor  defects  found  per  inspection  prior  to  the  AWO  intervention  was  2.6, 
compared  to  4.2  minor  defects  found  after  the  AWO  intervention,  resulting  in  a  36.5% 
increase.  The  increase  in  minor  discrepancies  (1.6)  found  during  follow-up  assessments 
conducted  after  the  AWO  intervention  are  indicative  of  the  trends  in  the  process 
measures  listed  above  (QVR  ratings,  score). 

Tinker  Results 

Pretest  and  post-test  data  sets  for  Tinker  AFB  also  reflect  mean  QVR  ratings, 
Quality  Assurance  scores,  and  number  of  major  and  minor  defects  noted  per  inspection. 
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Consideration  must  be  given  to  the  number  of  samples  that  comprise  the  data  sets.  As 
previously  mentioned,  the  number  of  pretest  samples  (9)  is  considerably  less  than  the 
number  of  post-test  samples  (66).  Baseline  estimates  of  pretest  quality  may,  as  a 
consequence,  be  somewhat  imprecise. 

AWO  Summary.  The  Action  Workout  effort  on  the  E-3  Isochronal  inspection 
process  occurred  from  4  November  to  8  November  1996.  The  critical  need  to  improve 
the  unit's  E-3  isochronal  inspection  process  at  Tinker  was  difficult  to  ascertain  due  to  two 
dramatic  yet  unrelated  process  changes  that  took  place  prior  to  the  AWO  event.  The  unit 
had  already  realized  significant  improvements  in  aircraft  maintenance  down  time  by 
changing  the  maintenance  inspection  concept  from  phase  to  isochronal  and  through  the 
acquisition  and  implementation  of  maintenance  scaffolding.  The  phase  to  isochronal 
conversion  process  enabled  the  unit  to  reduce  non-mission  capable  time  by  120  days  per 
year.  The  use  of  scaffolding  enabled  the  Wing  to  eliminate  294.8  hours  per  phase 
inspection.  When  the  AWO  team  arrived  at  Tinker  AFB  on  8  Nov,  the  E-3  isochronal 
inspection  process  took  approximately  4.2  days.  After  conferring  with  process  owners, 
five  teams  were  assembled  to  focus  on  the  critical  path  tasks  that  offered  the  best 
opportunities  for  further  improvement.  Once  again,  eliminating  unnecessary  process 
steps,  consolidating  equipment,  reducing  travel  distance,  and  revising  work  sequences 
generated  significant  results.  According  to  the  AWO  team  chief,  enthusiasm,  and  a  "can- 
do"  spirit  was  displayed  by  everyone  involved,  including  senior  leadership.  This  top 
level  commitment  and  employee  "buy-in"  are  deemed  as  equally  important  to  the  AWO 
improvement  effort  as  the  improvement  ideas  and  innovations  created  by  team  members. 
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The  report  indicated  that  a  full  day  was  eliminated  from  the  E-3  inspection  process 
(Action,  1997). 


Table  10.  Tinker  AFB  QVR  Ratings 


Pretest3 

Post-testb 

X2 

Variables 

Freq 

% 

Freq 

% 

(df) 

QVR  1 

3 

37.50 

46 

76.70 

*5.38 

1 

QVR  3 

5 

62.50 

14 

23.30 

aN  =  8 
bN  =  60 
*  p  <  .05 


QVR  Ratings.  Table  1 0  illustrates  the  frequency  and  percentage  of  only  the 
QVR  1  and  QVR  3  ratings  that  complete  the  pretest  and  post-test  data  sets  at  Tinker 
AFB;  QVR  2  ratings  are  not  shown.  Pretest  data  indicated  that  3  of  9  inspections 
received  a  QVR  1  rating  (33.3%),  1  inspection  received  a  QVR  2  rating  (11.1%),  and  5 
inspections  received  QVR  3  ratings  (55.6%).  Post-test  data  showed  46  QVR  1  ratings 
(69.7%),  6  QVR  2  ratings  (9.1%)  and  14  QVR  3  ratings  (21.2%).  In  using  the  chi  square 
analysis,  the  pretest  QVR  2  rating  produced  an  expected  value  of  .84.  Since  the  expected 
value  was  less  than  one,  the  chi-square  analysis  could  not  be  performed  properly  (Cooper 
and  Emory,  1995:  452-453).  Therefore,  all  QVR  2  ratings  were  removed  from  further 
analysis.  Results  indicate  a  39.2%  increase  in  the  number  of  QVR  1  ratings  received 
after  the  AWO  intervention.  Chi-square  analysis  indicated  that  the  difference  between 
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QVR  ratings  was  statistically  significant  (X2  =  5.3782,  df=  1).  The  increase  in  QVR  1 
ratings  is  substantial. 


Table  11.  Tinker  AFB  Quality  Assurance  Scores 


Pretest3 

Post-testb 

Variable 

Mean 

S.D. 

Variance 

Mean 

S.D.  Variance 

T-stat 

Score 

8.89 

5.97 

35.61 

4.63 

4.26  18.14 

*-2.67 

a  N  =  9 
h  N  =  66 
*  p  <  .05 


Quality  Assurance  Scores.  The  mean  score  for  the  pretest  data  set  was  8.9.  The 
mean  score  for  the  post-test  data  set  was  4.6.  Results  indicate  a  48.3%  decrease  in 
overall  score.  T-test  analysis  for  two  independent  samples  indicated  that  the  difference 
between  mean  scores  was  statistically  significant  (p=.0093).  The  increase  in  Quality 
Assurance  scores,  as  shown  in  Table  1 1,  is  also  substantial. 


Table  12.  Tinker  AFB  Major  and  Minor  Defects 


Pretest3 

Post-testb 

Variable 

Mean 

S.D. 

Variance 

Mean 

S.D. 

Variance 

T-stat 

Major 

1.00 

1.12 

1.25 

0.35 

0.75 

0.57 

-1.70 

Minor 

4.89 

2.15 

4.61 

3.24 

0.35 

8.25 

-1.65 

aN=  9 
bN  =66 
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Major  Defects.  Prior  to  the  AWO  intervention,  an  average  of  1  major  defect 
was  found  per  inspection,  compared  to  .4  major  defects  found  per  inspection  after  the 
intervention  (see  Table  12).  Although  results  indicated  a  decrease,  the  difference 
between  the  mean  number  of  major  defects  in  the  pretest  and  post-test  data  sets  was  not 
significant. 

Minor  Defects.  The  average  number  of  minor  defects  found  per  inspection  prior 
to  the  AWO  intervention  was  4.9,  compared  to  3.2  minor  defects  found  after  the 
intervention,  resulting  in  a  34.7%  decrease.  Again,  the  difference  was  not  significant. 

Summary 

Favorable  results  or,  indications  of  improvement,  were  found  in  nearly  all 
inspection  process  measures  at  Tinker  AFB  and  Offutt  AFB.  Quality  Assurance  scores 
and  QVR  ratings  at  these  two  units  showed  significant  differences  between  pretest  and 
post-test  data  sets.  Comparatively,  Davis  Monthan  AFB  yielded  opposite  trends  in  every 
process  measure  except  for  the  number  of  major  defects.  However,  the  data  indicated 
that  differences  among  these  process  measures  were  not  statistically  significant.  The 
only  significant  difference  was  the  increase  in  the  number  of  minor  defects  found  by 
Quality  Assurance  after  the  AWO  intervention.  This  signifies  that,  on  post  AWO 
inspections,  inspection  section  maintenance  personnel  are  failing  to  identify 
approximately  1 .6  (36.5%)  fewer  minor  defects  than  were  found  prior  to  the  AWO 
intervention. 
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V.  Conclusion 


Overview 

The  objective  of  this  study  was  to  gather  and  analyze  aircraft  inspection  data  to 
determine  if  quality  enhancements  were  realized  after  an  Action  Workout  was 
accomplished.  Three  units  were  identified  as  having  major  aircraft  inspection  processes 
improved  by  the  AWO  concept.  Pretest  and  post-test  assessment  data  from  the  units 
were  analyzed  to  determine  whether  overall  quality  improvements  were  made. 

This  study  operationally  defined  quality  in  terms  of  Quality  Verification 
Assessment  ratings  compiled  before  and  after  each  AWO  event.  The  Quality 
Verification  Assessment  criteria  used  in  this  study  were  QVR  ratings,  Quality  Assurance 
scores,  and  the  number  of  major  and  minor  defects  found  by  Quality  Assurance 
inspectors  during  follow-up  assessments  of  inspection  section  personnel.  Comparisons 
were  made  to  determine  if  ratings  improved,  declined,  or  remained  unchanged. 
Parametric  t-tests  were  used  to  determine  the  significance  of  any  differences  between  the 
sample  means  (Quality  Assurance  scores,  majors,  and  minors),  as  applicable,  and 
nonparametric  chi-square  analyses  were  used  to  determine  differences  between  QVR 
ratings.  This  chapter  will  provide  a  summary  and  discussion  of  findings,  highlight 
limitations,  and  recommend  future  research  directions. 

Summary  and  Discussion 

This  study  investigated  whether  drastic  process  improvements  brought  about  by 
Action  Workouts  have  any  effect  on  process  quality  as  a  whole.  Overall,  the  results 
provide  at  least  plausible  evidence  suggesting  that  quality  enhancements  can  be  realized 
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as  a  result  of  Action  Workouts.  Results  at  two  of  the  three  units  analyzed  indicate  that 
overall  quality  of  major  aircraft  inspection  processes  improved  considerably,  quite 
possibly  as  a  result  of  the  Action  Workout  intervention.  QVR  ratings  (considered  as  the 
most  relevant  process  measure)  and  overall  Quality  Assurance  scores  dramatically 
improved  at  both  Offutt  AFB  (2 6.6%  increase)  and  Tinker  AFB  (39.2%  increase),  but 
remained  essentially  unchanged  at  Davis  Monthan  AFB  (3.5%  decrease).  The  "dramatic" 
increases  at  Offutt  and  Tinker  AFB's  were  found  to  be  statistically  reliable. 

The  evidence  also  suggests  that  many  of  the  changes  and  innovative  process 
improvement  ideas  reported  by  the  Air  Combat  Command  Quality  and  Management 
Innovation  Squadron  are  seemingly  intact  and  being  utilized  by  respective  maintenance 
personnel  at  Offutt  AFB  and  Tinker  AFB.  This  is  important  in  that  it  indicates 
acceptance  of  change,  and  employee  "buy-in, "  two  essential  principles  of  quality 
improvement.  It  may  also  be  inferred  that  leadership  commitment  exists  at  these  two 
units.  This  is  not  to  say  that  this  "quality  culture"  is  lacking  at  Davis  Monthan  AFB,  but 
rather,  the  available  data  made  it  difficult  to  make  any  inferences  whatsoever. 

Virtually  no  significant  difference  was  found  between  the  mean  number  of  major 
defects  in  the  pretest  and  post-test  data  sets  at  Tinker  AFB  and  Davis  Monthan  AFB. 
Evidence  showed  decreasing  trends,  however,  at  both  units. 

The  only  statistically  reliable  difference  that  indicated  a  decline  in  quality  was 
found  in  the  minor  defect  samples  at  Davis  Monthan  AFB.  An  increase  in  the  number  of 
minor  defects  were  found  when  comparing  pretest  and  post-test  data  sets.  Essentially,  on 
post-AWO  inspections,  Davis  Monthan  maintenance  personnel  were  failing  to  identify  an 
average  of  1 .6  minor  defects  fewer  per  phase  inspection  than  they  did  prior  to  the  AWO 
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intervention.  This  statistically  significant  difference  is  congruent  with  the  trends  in  both 
Quality  Assurance  scores  and  QVR  ratings,  but  is  not  considered  to  be  substantial.  It  is 
possible  that  some  of  the  changes  brought  about  by  the  AWO  either  hampered  the 
inspection  process,  or  some  changes  may  not  have  been  fully  integrated  into  the  process. 
The  available  data  does  not  provide  explanations  for  this  downward  trend. 

Limitations 

The  ex-post  facto  research  design,  along  with  the  limited  amount  of  pretest  and 
post-test  data,  provided  weak  scientific  measurement  power  for  this  study.  Several 
threats  to  both  internal  and  external  validity  exist  (Cooper  and  Emory,  1995:  358-361). 

In  interpreting  the  results  of  this  study,  it  must  be  noted  that  many  extraneous  variables 
exist  that  may  conceivably  affect  the  relationship  between  AWO's  and  process  quality, 
further  lessening  the  support  for  causality.  A  crucial  limitation  of  this  study  was  that 
available  data  provided  little  insight  for  determining  explanations  for  relational  effects 
that  were  found. 

This  study  did  not  examine  self-report  variables,  such  as  experience,  proficiency, 
job  satisfaction,  or  stress.  Experience  and  proficiency  of  maintenance  personnel,  both  of 
which  contribute  to  job  performance  measures  or  "quality"  of  work,  may  have  changed 
over  the  period  of  the  study.  Similarly,  the  experience  and  proficiency  of  the  Quality 
Assurance  personnel  who  performed  the  assessments  may  have  also  changed.  Generally, 
an  experienced  and  technically  competent  evaluator  will  be  able  to  identify  more 
discrepancies  while  doing  an  assessment  than  an  evaluator  of  lesser  proficiency  and 
experience.  The  effects  of  job-satisfaction,  morale,  and  stress  may  have  also  changed 
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over  the  study  time  period.  Although  force  drawdowns  have  somewhat  stabilized  over 
the  past  year  in  the  Air  Force,  the  operations  tempo  continues  to  escalate.  This  increased 
"ops  tempo"  undoubtedly  affects  overall  job  performance,  morale,  and  stress,  especially 
in  the  maintenance  field. 

This  study  was  also  limited  by  the  amount  of  variables  used  to  define  the  quality 
construct.  This  study  used  available  archival  assessment  data  as  the  only  measure  of 
quality.  With  the  exception  of  inspection  days,  the  assessment  measures  were 
nonorthogonal.  For  example,  the  number  of  defects  found  during  an  assessment 
determines  the  score,  and  the  score  determines  the  QVR  rating.  Supplemental  measures, 
such  as  customer  satisfaction  data  and  aircraft  reliability  statistics,  might  improve  the 
scientific  measurement  power  of  this  study. 

As  previously  mentioned,  historical  process  data  (man-hours,  cycle  time,  distance 
traveled,  etc.,)  were  virtually  nonexistent,  except  for  the  data  reported  in  the  applicable 
AWO  trip  reports.  Therefore,  it  was  difficult  to  ascertain  specific  reasons  for  any 
relationships  that  were  identified. 

Yet  another  limitation  of  this  study  was  that  it  focused  on  an  extremely  complex 
process-major  aircraft  inspections.  Analysis  of  less  complicated  processes  that  have 
undergone  AWO  interventions  may  yield  more  powerful  results.  These  limitations  also 
make  it  difficult  to  generalize  results. 

Suggestions  for  Future  Research 

To  the  best  of  my  knowledge  this  is  the  only  study  that  has  explored  the 
ramifications  of  Action  Workout's.  Albeit  limited,  this  study  has  set  the  foundation  for 
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further  research  in  this  area.  Further  research  is  important  so  that  Air  Force  leaders  are 
able  to  make  informed  decisions  about  how  best  to  utilize  this  improvement  tool. 

As  previously  mentioned,  future  studies  assessing  the  effectiveness  of  AWO's  are 
necessary.  Longitudinal  studies  could  be  made  to  assess  AWO  process  measures,  such  as 
cycle  time,  distance-traveled,  and  man-hours.  This  is  crucial  for  the  Air  Force  because 
the  studies  might  provide  better  insight  as  to  what  "works,"  what  doesn’t  work,  and  more 
importantly,  why.  It  would  also  indicate  whether  AWO  improvement  ideas  and 
innovations  remained  in  effect  over  extended  time  periods. 

Another  follow-on  study  may  attempt  to  evaluate  the  overall  quality  culture  in 
similar  pretest,  post-test  scenarios.  This  may  further  support  causal  inference  between 
AWO's  and  their  overall  effectiveness. 

Conclusion 

According  to  the  Air  Combat  Command  1998  Strategic  Plan,  the  two  major 
challenges  facing  Air  Combat  Command  are  a  high  operations  tempo  and  operating  in  a 
"fiscally  austere  environment"  (Action,  1997).  Specific  goals  were  identified  by  senior 
leaders  to  meet  these  challenges  head-on.  One  of  the  goals  for  ACC  is  to  reduce 
operating  costs  by  30  %.  Action  Workouts,  designed  to  achieve  savings  in  manpower 
and  time,  have  been  identified  as  one  of  the  primary  means  for  achieving  this  goal.  This 
study  has  established  some  plausible  evidence  that,  in  addition  to  the  reported  manpower 
and  time  savings,  Action  Workouts  may  enhance  overall  process  quality.  The 
implications  for  the  Air  Force  are  critical  in  that  continuous  improvement  initiatives  are 
essentially  becoming  the  means  for  "operationalizing  quality"  in  the  Air  Force  today. 
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